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Context: 

Evidence of persistent differences in productivity across firms can be found in almost 
every sector of the economy (Bloom & Van Reenen, 2011). In education, there is ample 
evidence that some schools far outperform others at raising student achievement even when 
accounting for differences in the students they serve and the resources at their disposal. 
Differences in the human capital stock of teachers across schools cannot fully account for the 
differential productivity across schools. In teaching, as in any occupation where professionals 
perform their work in organizational contexts, productivity is influenced by both individual and 
organizational factors (Hackman & Oldman, 1980; Kanter, 1983; Johnson, 1990). A growing 
body of literature attempts to identify, measure, and evaluate the potential contribution of 
organizational practices and contexts to overall productivity in schools and in the private sector 
(Simon & Johnson, forthcoming; Lazear and Oyer, 2009). However, this literature has been 
largely limited to cross-sectional analyses and longitudinal case studies of individual 
organizational (Bloom & Van Reenen, 2011). 

The proliferation of school surveys administered to teachers, students and/or parents 
presents a unique opportunity to capture large-scale data on organizational contexts. Although 
some organizational features, such as school resources, can be readily measured with 
administrative data, school surveys now allow researchers to characterize the ways in which 
administrators, teachers and students interact within schools. These surveys provide important 
data on the organizational culture and context of schools, but their utility to researchers has been 
constrained by their infrequent administration and low response rates in most districts. 

Purpose: 

This study is among the first to address the empirical limitations of prior studies on 
organizational contexts by leveraging one of the largest survey administration efforts ever 
conducted in the United States outside of the decennial population census. Since 2007, the New 
York City Department of Education (NYC DOE) has administered an annual school surveys to 
teachers, parents, and students with consistently high response rates. We identify distinct, 
malleable dimensions of middle schools’ organizational contexts using teachers’ responses to the 
annual School Survey and estimate the relationship between these measures and teacher 
turnover, and growth in students’ academic achievement. Our panel data allows us to identify 
these estimates using within-school variation over time by accounting for any time-invariant 
differences across schools and unobserved time-shocks through school and year fixed effects. 

Setting & Sample: 

We focus our analyses on NYC middle schools for several reasons. First, the middle 
grade years are crucial in students’ academic and social-emotional development and play a 
critical role in influencing students’ high school and post-secondary outcomes (Balfanz, 2009). 
Despite this, middle schools have uncommonly high rates of teacher turnover (Marinell & Coca, 
2013), teachers often consider middle school assignments as less desirable than comparable 
elementary or high school assignments, and middle school teachers may receive less tailored 
preparation than elementary and high school teachers (Neild, Farley-Ripple & Byrnes, 2009). 

We create three primary analytic datasets to conduct our analyses of NYC middle 
schools. First, we construct a teacher-year level panel dataset that contains teachers’ responses 
to the School Survey. Our analytics sample spans five years from 2007-08 and 2011-12. As 
shown in Table 1, response rates were 63 percent in 2007-08 and rose incrementally each 
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following year to 83 percent by 201 1-12. Using a teacher-year level panel dataset built from 
human resources files, we create school-level averages of teacher turnover and teacher 
characteristics. Importantly, this panel includes data through 2012-13 which allow us to create 
measures of teacher turnover for the 2011-12 academic year. We restrict this dataset to include 
only full-time middle school classroom or special education teachers using the NYC DOE’s 
approach to identifying active teachers, resulting in 54,050 teacher- year observations and 16,408 
unique teachers. Our primary analytic dataset is a school-year panel from 2008 to 2012 that 
contains 1,151 school-year observations for 278 unique middle schools in the NYC DOE. We 
restrict our sample to include only middle schools with traditional grade 6-8 configurations. In 
Tables 2 and 3, we present descriptive characteristics of the teachers and schools in our sample. 

Measures: 

We construct two sets of primary outcomes for our regression analyses described below: 
school-level average measures of teacher turnover and median student growth percentiles. The 
average turnover rate among schools in our sample is almost 18% annually, with nearly 8% 
transferring to other schools and 10% taking non-teaching positions or leaving the NYC public 
school system. We estimate student growth scores in mathematics and ELA following the 
methods developed by Betenbenner (2009) and operationalized in the Student Growth Percentile 
(SGP) software package in R (Betebenner, 2011). 

We draw upon both theory-driven and data-driven approaches to inform our process of 
constructing school context measures. We first restricted the pool of items to only those that 
were included on each survey from 2008 to 2012. We then conducted a principal components 
analysis on our panel dataset of teachers’ responses to the School Survey to identify the number 
of distinct factors captured by our 33 items. A four factor solution suggested by our visual 
analyses of the marginal decrement in eigenvalues was further reinforced by applying the Kaiser- 
Guttman stopping criterion. In order to construct measures of the four factors suggested by our 
exploratory analysis, we conducted a principal-component factor analysis, followed by an 
orthogonal (varimax) rotation. We calculate factor scores for each teacher in each year, and then 
average these scores to the school-year level to obtain our primary predictors. Finally, we 
standardize each of these school context measures in a school-level dataset to facilitate the 
comparison of the relative magnitudes of their associated coefficients in our analyses. 

Research Design: 

We analyze the relationship between our two primary outcomes of interest, teacher 
turnover and student growth, and measures of school context in a school-year dataset using two 
classes of Generalized Linear Models (GLMs). This approach best reflects the school-level 
processes we hope to capture and the level at which our standard errors are most appropriately 
estimated. Formally, analytic approaches can be expressed using the following general model: 

(1) #{£(3^)} = /?' Dimension s t + r( T st + <p' S st + n 3 + y t 

where E(T 3( .) is the expected value of the outcome for school s in year t, conditional on 
observable predictors; g[ m } is a link function; Dimensions 3t is the vector of school-context 
factors described above; T„ t is a vector of teacher characteristics, averaged within school and 
year; S 3t is a vector of school characteristics; and n 3 and y t are fixed effects specific to school 
and year, respectively. Average teacher characteristics include controls for gender, race, 
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experience, and degrees. School characteristics include controls for the proportion of students by 
gender, race, free/reduced lunch eligibility, special education status, and English language 
learner status, as well as log enrollment and an indicator for schools that provide free lunch to all 
students (i.e., universal feeding schools). For teacher turnover, we set #{■} to be the logit 
function. For student growth percentiles, we take Y 3t to be the median student growth percentile 
score in mathematics or ELA and set g{ - } as the identity function. We assume these medians to 
be approximately normally distributed about their conditional means, resulting in an Ordinary 
Least Squares model. 

Findings: 

Four Dimensions of the School Context: We construct four distinct measures of the 
school context based on our analyses of teachers’ responses to the School Survey: leadership & 
professional development ( Leadership ), high academic expectations for students ( Expectations ), 
teacher relationships and collaboration ( Relationships ), and school safety & order {Safety). 

These four dimensions aligned closely with the thematic groupings that our research team 
independently identified prior to conducting our factor analyses. We arrived at these four 
dimensions by characterizing the dominant items on each factor with rotated factor loadings of 
0.6 or higher. As shown in Table 4, the Leadership dimension has the most items with large 
factor loadings and explains 21 percent of the variance across our panel of teacher responses. 

In order to provide readers with intuitive measures of how NYC DOE middle school 
teachers perceive the school contexts in which they work, we calculate the percentage of teachers 
who either agree or strongly agree with each survey item in a given school-year. On average, we 
find that teachers view their school favorably; the mean agreement rate across school-years to 
each School Survey item asking about positive aspects of the school climate was over 70%. 
Overall, teachers were most likely to agree with the items that contribute to the academic 
expectations dimension, followed closely by the teacher relationships & collaboration dimension. 
Teachers were less readily willing to agree that their school benefited from high quality 
leadership and professional development and that they taught in safe and orderly schools. 

Teacher Turnover: We estimate the relationship between school context measures and 
our outcomes of interest using models both without and with school fixed effects as well as with 
all measures included separately and simultaneously. This produces four set of results for each 
outcome of interest and helps to illustrate important differences across estimation strategies. Our 
baseline model omits school fixed effects and estimates the relationship between individual 
dimensions of the school context, entered separately in each model, and a given outcome. We 
then re-estimate these models that draw on both within- and between-school variation, but 
include all four school context measures simultaneously. Our third set of analyses returns to 
estimating school context measures separately, but restricts our identifying variance to within- 
school variation over time by adding school fixed effects. Finally, we fit our preferred 
specification which includes all measures simultaneously with school fixed effects. 

In Table 5, we present predicted marginal effects of a one standard deviation increase in 
measures of the school context with teacher turnover. Our Generalized Linear Model relates 
school context dimensions to teacher turnover via an S -shaped logistic curve, which allows the 
marginal relationship to differ across the distribution of our predictors. We present estimates of 
these marginal effects at the 10 th , 50 th , and 90 th percentile of the distribution of each school 
context measure to characterize this curvilinear relationship. The results we present in Table 5 
illustrate four important findings: 1) we replicate previous findings that schools with higher 
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quality contexts experience lower turnover, 2) we show that improvements in the organizational 
context within a school over time are associated with corresponding decreases in teacher 
turnover, 3) we find that increases in school context measures have larger marginal effects on 
turnover for schools that start at lower levels of school context quality, and 4) we document the 
independent relationship between multiple dimensions of the school context and teacher 
turnover. 

Student Achievement Growth: In Table 6, we present a parallel set of estimates to those 
described above but exchange our teacher turnover outcome for measures of the median student 
growth percentile in mathematics or English language arts. Our results reveal four key findings: 
1) we replicate and extend previous research which finds that schools with higher quality school 
contexts have students who experience greater growth in achievement, 2) we show that 
improvements in the school context within a school over time are associated with corresponding 
increases in student achievement growth, 3) we find that the relationship between the school 
context and the median Student Growth Percentile in at a school is stronger in mathematics than 
in ELA, and 4) we illustrate that the relationship between the school context and student growth 
varies considerably across dimensions. 

Threats to Validity: The analytic methods discussed above allow us to show clearly that, 
among NYC middle schools, improvements in the school context within a school over time are 
associated with corresponding decreases in teacher turnover and increases in student 
achievement growth. Ultimately, we want to know whether the changes we observe in school- 
specific turnover rates and student growth are caused by changes in the school context. Our 
research design does not allow us to estimate this causal relationship conclusively because we 
lack a source of exogenous variation in our measures of the school context. Recognizing this 
limitation, we attempt to better understand the underlying relationship we observe between 
school context dimensions, turnover, and achievement by examining the most plausible 
alternative explanations for the patterns in our data. Robustness tests on a range of potential 
threats including common source bias, survey non-response bias, reverse causality, and omitted 
variable bias reveal no evidence that can explain away the relationships we find. 

Conclusions: 

While it seems intuitively obvious that an individual’s performance is affected by the 
environment in which she works, policymakers have largely overlooked the central role that 
schools play in influencing teachers’ in their push to overhaul teacher evaluation systems. In 
instances where aspects of school context — e.g., a principal who is an ineffective instructional 
leader, a school that lacks a consistent disciplinary code — are partly, or largely, responsible for 
languishing student performance, efforts to measure and strengthen the performance of 
individual teachers are unlikely to be adequate remedies in themselves. For teachers who have 
the misfortune of trying to deliver high-quality instruction and improve their craft amidst 
organizational dysfunction, continually reshuffling the staff in search of teachers who can be 
successful in spite of organization limitations is likely be an ineffective strategy to improving 
instructional quality. To systematically improve student performance, school and district leaders 
need robust evidence about the strengths and weaknesses of both individual teachers and the 
school organization as a whole. Equipped with this data, policymakers and practitioners can take 
steps to address individual as well as organizational strengths and deficiencies. 
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Appendix B. Tables and Figures 


Table 1: Teacher Response Rates to the NYC DOE School Survey 



Average 

25th Percentile 

75th Percentile 

2008 

63 

48 

80 

2009 

75 

62 

92 

2010 

78 

68 

93 

2011 

82 

72 

95 

2012 

83 

75 

95 


Table 2: New York City Middle School Teacher Characteristics 


Female 0.696 

Asian 0.053 

Hispanic 0.225 

African American 0.128 

White 0.579 

Novice 0.053 

2-3 years experience 0.119 

4-10 years experience 0.569 

1 1-20 years experience 0. 128 

> 20 years experience 0.131 

Bachelor's Degree 0.149 

Master's Degree 0.428 

Master's Degree plus 30 Credits 0.420 

Transfer 0.062 

Leave 0.089 

Turnover 0.151 


Notes : Sample includes 54,050 teacher-years and 16,408 unique 
teachers 
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Table 3: New York City School Characteristics 



mean 


Percentile 


25 th 

50 th 

75 th 

Enrollment 

639 

280 

477 

939 

% Free/Reduced Price Lunch Eligible 

82.64 

75.86 

88.79 

99.43 

% Special Education 

11.63 

8.06 

11.17 

14.95 

% English Lanuge Learner 

13.07 

4.44 

9.23 

18.09 

% African American 

34.66 

10.76 

26.12 

60.08 

% Asian 

11.31 

0.86 

2.88 

14.61 

% Hispanic 

43.06 

18.83 

41.02 

65.81 

% White 

10.55 

0.64 

1.69 

14.77 

% Teachers Transfer Schools 

7.84 

1.75 

5.26 

11.11 

% Teachers Leave Teaching 

10.00 

5.33 

8.70 

13.33 

% Turnover 

17.84 

9.59 

15.38 

23.53 

SGP: Math 

47.98 

39.00 

48.00 

57.00 

SGP: ELA 

47.42 

41.00 

47.50 

54.00 

Notes: Sample includes 1151 schools years and 278 unique schools 


Table 4: Descriptive Statistics for Four School Context Factors 


Number of items 


Proportion of 


with loadings 

Eigen Value 

Variance 


over 0.5 


explained 

Leadership & Professional Development 

11 

7.00 

0.21 

Academic Expectations 

6 

5.93 

0.18 

Teacher Relationships & Collaboration 

5 

4.45 

0.14 

Safety & Order 

5 

3.69 

0.11 


Notes: Eigen values and proportion of variance explained are from exploratory factor analyses 
with a four factor solution, and a varimax orthogonal rotation. 
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Table 5: Predicted Marginal Effects of Four Dimensions of the School Context on Teacher Turnover 



Percentile of the school context measure at which the predicted marginal 
effect is estimated 

10th 50th 90th 10th 50th 90th 

(la) 

(ib) 

(lc) 

(2a) 

(2b) 

(2c) 


Panel A: Regression estimated for for each factor separately 

Leadership & Professional Development 

-0.022** 

-0.020*** 

-.018*** 

_ 019*** 

_ 017*** 

-.016*** 


(0.003) 

(0.002) 

(0.002) 

(0.004) 

(0.003) 

(0.003) 

Academic Expectations 

-.021*** 

_ 019*** 

- 017*** 

-.009* 

-.008* 

-.008* 


(0.003) 

(0.003) 

(0.002) 

(0.004) 

(0.003) 

(0.003) 

Teacher Relationships & Collaboration 

-.007* 

-.006* 

-.006* 

-.007* 

-.006* 

-.006* 


(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

Safety & Order 

- 023*** 

-.021*** 

-.018*** 

-.013** 

-.013** 

-.012** 


(0.004) 

(0.003) 

(0.002) 

(0.005) 

(0.004) 

(0.004) 


Panel B: Factors included simultaneously in each regression 

Leadership & Professional Development 

- 014*** 

-.013*** 

-.012*** 

-.016*** 

-.015*** 

- 014*** 


(0.003) 

(0.002) 

(0.002) 

(0.004) 

(0.003) 

(0.003) 

Academic Expectations 

_ on*** 

-.010*** 

-.010*** 

-0.005 

-0.004 

-0.004 


(0.003) 

(0.003) 

(0.002) 

(0.003) 

(0.003) 

(0.003) 

Teacher Relationships & Collaboration 

-.006* 

-.006* 

-.006* 

-.007* 

-.007* 

-.007* 


(0.003) 

(0.003) 

(0.002) 

(0.003) 

(0.003) 

(0.003) 

Safety & Order 

-.015*** 

_ 014*** 

-.013*** 

-.009* 

-.008* 

-.008* 


(0.003) 

(0.003) 

(0.002) 

(0.004) 

(0.004) 

(0.004) 

Year EE 


Yes 



Yes 


School EE 


- 



Yes 


n 


1151 



1151 



Notes: ***p<0.001, **p<0.01, *p<0.05. Standard errors reported in parentheses are clustered by school. Estimates 
are derived from generalized linear models with a logit link function. All models include vectors of teacher and 
student characteristics. Teacher-year level covariates include controls for average teacher gender, race, experience 
and degrees. School-year level covariates include controls for enrollment (logged) and average student gender, race, 
free/reduced-price lunch status, special education status, and English language learners. 
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Table 6: The Relationship between Four Dimensions of the School Context and School Median Student 
Growth Percentiles in Mathematics and English Language Arts 

Mathematics (SD=12.6) English Language Arts (SD=8.7) 

(1) (2) (3) (4) 

Panel A: Regression estimated for each factor separately 


Leadership & Professional Development 

1.063* 

1.162* 

.778** 

0.178 


(0.478) 

(0.533) 

(0.289) 

(0.327) 

Academic Expectations 

2.906*** 

1.804*** 

1.637*** 

0.204 


(0.459) 

(0.505) 

(0.325) 

(0.361) 

Teacher Relationships & Collaboration 

0.109 

0.44 

-0.175 

- 0.111 


(0.471) 

(0.517) 

(0.313) 

(0.449) 

Safety & Order 

5.208*** 

3.337*** 

2.788*** 

1.137* 


(0.586) 

(0.698) 

(0.371) 

(0.486) 


Panel B: Factors included simultaneously in each regression 

Leadership & Professional Development 

-0.619 

0.456 

-0.136 

-0.047 


(0.465) 

(0.546) 

(0.285) 

(0.339) 

Academic Expectations 

2.187*** 

1.597** 

1240 *** 

0.186 


(0.437) 

(0.520) 

(0.304) 

(0.365) 

Teacher Relationships & Collaboration 

0.032 

0.539 

- 0.211 

-0.047 


(0.443) 

(0.495) 

(0.313) 

(0.448) 

Safety & Order 

4.875*** 

3.189*** 

2.513*** 

1.138* 


(0.599) 

(0.736) 

(0.376) 

(0.511) 

Year FE 

Yes 

Yes 

Yes 

Yes 

School FE 

- 

Yes 

- 

Yes 

n 

1151 

1151 

1148 

1148 


Notes: ***p<0.001, **p<0.01, *p<0.05. Standard errors reported in parentheses are clustered by school. 
Estimats are derived fromOLS regression models. All models include vectors of teacher and student 
characteristics. Teacher-year level covariates include controls for average teacher gender, race, experience 
and degrees. School-year level covariates include controls for enrollment (logged) and average student 
gender, race, free/reduced-price lunch status, special education status, and English language learners. 
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